participants talk about the cybersecurity threat landscape, harnessing artificial intelligence and enhancing cybersecurity infrastructure. one of the participants was the federal chief information security officer who oversees security for the entire u.s. government. it's an hour and 45 minutes. >> good afternoon. welcome to the tenth annual billington cybersecurity summit. it's great to see so many friendly faces and such a great audience today. my name is tom billington, the chair of today's conference. it's an honor and privilege to convene this forum for a milestone tenth year to address our high purpose and theme, a call to action for the cybersecurity community. in this room, as you look

around, both in the audience here and throughout the exhibit hall, are some of the most innovative cybersecurity companies and organizations in the world facing some of the toughest adversaries. we thank you for your dedication to this mission. my wonderful wife, susan, founded billington cybersecurity nearly ten years ago. besides this annual summit we host the leadership council, a membership council. we aim to be top experts in the serious and deep dialogue about cybersecurity in our nation's capital. i deeply thank the superb speakers who will share their insights. please let's give them a round of applause. [ applause ] for the media, including cspan that's filming the event today, this conference is on the record unless specified and unclassified. and we welcome those members of

the media today. you can follow us on twitter at billington cyber. and use #billingt#billingtonsum. we have a packed day and a half ahead. it has been expanded by a half day this year. q&a will be available for some but not all sessions. either by live mic or nobilitec. i'd like to thank our partners and exhibitors who make the event possible today. and they really do. without them, we could not host this program. i'd like to thank them beginning with our lead underwriter, northrup grumman. allen hamilton, our diamond sponsors, google cloud, aws, cisco, hp federal. at platinum, mcafee, bit site

and blt. and at bronze, ativo networks. i also want to mention that we also have three country zones this year, which we're very excited about as well. to my left we have the uk's cyber innovation zone which is in its fifth year. to my right, we have the israel innovation zone and the canada zone to my right which is in its third year. with that said,b we again appreciate all our exhibitors and partners, including our continuing education partners. please let's give them all a round of applause. [ applause ] one quick logistics note.

if you're an isc squared member, we do have continuing education for the first year. so please do go to the registration desk and give them your member number. and they'll be able to send you a digital certificate. so now it's my great honor to introduce for the first year a master of ceremonies for our program. known most -- to most in this room, captain ed diviny, recently retired after 34 years of service and most recently as the director for corporate partnerships and technology outreach at u.s. cybercommand. thanks to each of you for being here, have a great one and a half days. i'll be popping back on stage from time to time throughout the day and a half. but you're in extremely capable hands with captain ed diviny, my great friend and who i'm very honored to introduce to you now as our master of ceremonies. [ applause ]

>> hello, everyone and good afternoon. thank you very much for the very kind introduction and for the opportunity. we've been friends a long time and i'm honored to serve as your master of ceremonies here today. you and susan have built a great company that provides a much needed venue to discuss the most pressing cyberchallenges facing both corporations and our government. i'm excited about the great lineup of speakers today and the robust agenda ahead of us. enough for me, let's get our day started. it's my honor to introduce our welcoming keynote speaker, grant schneider. thank you grant for opening the conference and the floor is now yours.

♪ ♪ >> it's a shorter walk than it is for the sound. good afternoon. first of all, i want to thank you everyone for being here today. someone beforehand told me that i'm the first speaker, so i need to bring a lot of energy and rile up the crowd. but i'm a policy guy, so i'm not sure that's sort of in my mantra. you might want a more operational person and i think they're later on this afternoon. i want to thank tom for having me here today. and the ability to talk to you a little bit about the roles and responsibilities of what we do within office management and budget. and i think it's really well connected to the theme for this tenth annual cybersecurity

summit. the theme being a call to action to address tomorrow's top cyber challenges. this is at the core of what we do within o&b. we're trying to help agencies address thaer tey're top future challenges and their challenges from yesterday that they haven't finalized yet. we do that in a number of ways. if you go and look at the guiding document for our organization, is the federal information security modernization act of 2014. in that, it assigned a number of responsibilities to the o&b director around cybersecurity and we carry those out on his behalf. and if you look at it, there's six or seven items that were assigned to three main functions. first and foremost is developing and overseeing the implementation of government wide cybersecurity policies. number two, is insuring that

agencies are protecting federal information systems and data, commiserate with the potential risk of harm of a compromise. think risk management, not all those other words i used. third is assuring that federal agencies are complying with government wide cybersecurity standards, be those things from the national institute of standards and technology, o&b guidance, laws, binding operational directives from the department of homeland security. wrr working with agencies and holding them accountable to be able to deliver on those. i want to talk about a few things we've done in the last year around each of those. around developing and overseeing the implementation of new cybersecurity policies, we have updated -- excuse me, we're about to update our trusted internet connection policy. this is about while those three things i listed that we do to

help agencies, that's the what they need to do. we also need to provide them tools and capabilities from a broader government standpoint for their ability to actually deliver on those requirements. and so we're putting out a new policy. hopefully in a couple weeks around trusted internet connection. that is been out for public comment. so you've seen versions of it. but this is about how do we evolve our policies to adapt to technology changes and really the movement to cloud environments, which is absolutely critical as we look to moderninize federal information technology. secondly, last year the very end of the last congress, the president signed into law the federal acquwizition security a. we can have a federal acquisition security council and really look at the security of

the equipment that we're bringing into the federal space. there's a lot of work ongoing with that. but this really is a tool for federal agencies to be able to have a bit of a vetting of the equipment that's coming into their enterprise and be able to leverage both classified and unclassified information from making determinations they don't want to bring something into their environment. when we talk about protecting information, commiserate with the risk of potential compromise or risk management, it's all about risk management. we can't protect everything. we have to understand what is most cri most critical. we updated our policy at the beginning of this year. in addition, the department of homeland security updated guidance on high value assets. we've tried to partner with dhs to be able to provide a more

tactical level of input enin details for agencies to be married with or combined with the policy that we're putting out from an o&b standpoint. in addition to our hva update and really understanding what's most important to protect it, when it comes down to protecting our systems and our information, it's really a people challenge. and so our ability to have and your ability to have the right workforce, a capable cybersecurity workforce is absolutely critical. the president signed an executive order, which has a number of tasks, things we're really looking forward to for the federal enterprise around some cyber competitions. you'll hear more about hopefully in the coming months. also rotational programs. how can we rotate more and move of our cyber workforce from

agencies to agencies to grow the skills of those individuals, but also to enhance the abilities of other agencies and bring in outside talent. and then in addition to that is this year we launched our cybersecurity reskilling academic. we've had one cohort go through. we've got a second one goen. this is a pilot. the two cohorts are 50 or 60 people. this is about how can we take federal employees who are looking to move into another type of -- learn a new skill, learn cybersecurity and move into a new career, how can we leverage their dedication to the government, their understanding of what it takes to get stuff done in the federal enterprise and then teach them and train them in cybersecurity. they going to have enough they can apply on what they're working on and start to

transition kn transition into a new career path. how do we leverage those individuals. on the third one, which is insuring federal agencies comply with the variety of standards we have out there, we talk about compliance, it's used as a dirty word. i actually think though compliance is necessary but not sufficient. we have to have certain things out there that agencies need to comply with. we need to have some checklists. we need to be sure that agencies are taking advantage of the various tools and capabilities and resources that are available to them. and so, you know, as i mentioned, those come in the form of laws, memos, binding operational directives. as we move more into supply chain risk management, they'll come potentially in the form of removal and exclusion orders when we talk about equipment that can't be in the enterprise. obviously, a big yaer from

national institute of standards and technologies, special pubs and guidance we have and that they put out. and so today nist has released an update, and what this is about is about cyber resiliency. we're never going to prevent attacks, we're never going to stop bad guys from getting into our systems. how do we insure that we have resiliency of mission within cyberspace? i'd like to ask ron ross to come out and he's going to give you some of the highlights of this 800.160 rev 2 and i'm going to be back for a panel here in a few minutes, so thank you. >> thank you very much, grant. thanks to tom billington for giving us this opportunity to

announce a very important document. we've just finished this document about a week ago. it's been in development for about 18 months. and it really addresses some of the very difficult and challenging problems that we're all having today with regard to cybersecurity. if you recall the past several decades our strategy for protecting our critical assets has been a one dimensional strategy. stopping the bad guys at the front door before they get enand do damage. we know after many decades of empirical evident of the cyberattacks and things we've experienced, even when we do everything right, sometimes those high end adversaries find a way to get into the systems and compromise our critical assets. this addresses something call

cyber cyberresiliency. how can they take that punch and keep on operating even if it's in a debilitated status. it's our first attempt to extend that one dimensional cybersecurity protection to three dimensions. where the second dimension is called damage limitation. how do we limit the damage the adversaries can do once they've breached our systems? we assume that the adversaries are either in your system now or are getting in there at some point. the third dimension is going to be how do we make those systems cyberresilient? where they can continue to operate and are survivable. this document has a lot of practical guidance for all of our customers out there who want to take not only new development systems, systems that are going through that life cycle, but also the 95% of your systems that are legacy. how do you apply the techniques

and approaches for cyber resiliency to increase the level of protection for your critical assets and systems. this is a national imperative. we've seen over the last couple years, the adversaries are very capable, they're targeting our critical resources and doing great damage. for critical federal systems, voting systems to weapons systems, to power plant, cyberresileiancy is the wave of the future. we're trying to make these finite machines operate more like the human body with an immune system where you can get a cold or virus and then your immune system kicks in and it doesn't take you down completely. for the next 45 days, this final public draft will be on our website. we we encourage you to take a look

at our guidance. we have great use cases that deal with microgrids, enterprise, information technology systems, and there's a host of other things. we even have a couple of real world rotations on the cyberattack of 2015 and 2016 where we show how applies these constructs to your systems could stop some of these high end attacks by adversaries. thank you to tom billington for letting us have the time this morning. thanks to grant and all the folks at omb who have been very supportive. one last shoutout to all my team members who worked on this document non-stop, and also to the office of the vice president who have been very very supportive on helping move this guidance forward. we have a lot of critical defense systems. thank you very much and have a great canferenconference, folks.

appreciate it. >> thank you very much, grant and ron for the remarks. one programming note. for those of you who have been to our events in the past, we have an exhibition hall with a lot of vendors in a separate area. to be more inclusive and to allow a greater flow of communication, we chose to do everything all in one venue. if you would, please, because of that if you'd keep the conversations on the side down to a minimum to allow the speakers and those in the audience here to hear. so now, please let me, it's my honor to welcome the former deputy undersecretary for cybersecurity and communications at the department of homeland security.

she'll be leading a fireside chat with the only two people who have held the position of federal sisso. grant schneider and the retired general hill. >> thank you very much. good afternoon. and thank you all for being here, spending time with us on these important topics. i want to definitely thank the billington conference and the sponsors of course. i have 30 minutes to bring out -- it's almost unfair. only 30 minutes with the first federal chief security officer and our current federal chief information security officer doing great work. general, i'll start with you. it was a pleasure to work with you then. what was the highest impact areas you are working on? >> we take a look at the cybersecurity in the federal government, it's at learning continuum. we try to get better and build

upon the lessons learned from the past. we certainly tried doing that when i was in office. some of the more impactful things that we did, and i think grant is continuing with is first is changing the narrative and looking at cybersecurity as a risk management issue. previously not only in the public sector but the prevent sector we saw emphasis solely on just compliance. not necessarily taking a look at cybersecurity as a holistic risk management issue that involves people process and technology. so that's the first thing we've talked to paige for me. that was the narrative that we were trying to move forward on. i'm pleased to see that continuing. secondly, we were trying to make sure that we were in fact trying to implement best practices and identify them and sharing that. information sharing was

critically important. the ways we were doing that was through public/private partnerships and getting two-way communication between industry and the federal government. a lot of work that needs to be done on that. i think we really had an impact watching those programs and trying to get those best practices in place. i believe that compliance doesn't always bring you best practices, but best practices will always bring you compliance. the third thing i think was impactful was taking a look and making sure that we were best aligning technology with the mission needs. we launched the continuous d diagnostics program to try to raise the bar across the federal government. we had a lot of agencies that were large and well-funded then we had smaller agencies that weren't as well-funded and weren't as large but they still

had the same mission tasking to protect sensitive information. having the continuous diagnostic and mitigation program launched to help answer the questions of what's on my network, who is on my network and what's going on on my network across the federal government was a critical factor and success during our tenure. further making sure that that cdm program was available to state and local governments as well as to the dot domain was something i thought was a job well done by our team. >> thank you. if we look at the recent statistics, the work done by both of you shows the cdm, that program actually has improved the security of many of the federal agencies. so grant, you're now in the driver's seat. in that important position, how do you go forward because there's been a lot of progress made. you talked about partnering with o&b. >> i view it that we needed to

get a whole bunch of kind of base line policies in place and establish the ground floor of expectations for federal agencies. and that includes both the larger ones as well as the smaller ones that greg alluded to. really, though, where we're trying to focus on is how are we the maximum amount of assistance to agencies as they try to implement their cyber programs. the expectation is that every agency will be able to protect their information to the same degree. we expect the department of defense and the department of homeland security and the small business administration all to be able to do essential the same job. they're clearly not resourced similarly in order to do that. so we're trying to -- through partnerships with homeland security, yes, we have an oversight role and we do an amount of measures and measurement and holding accountability or holding agencies accountability.

but we want to be able to be there as a support structure. doing cyber staff rooeceviews. we come in, sit down with agencies. we work on particular problems to also bring solutions to those. whether it's solutions from another agency that's had a similar change or a solution or technical team for dhs. it's how how do we insure the huh doption adoption and the lencveraging o those. >> with the cyber strategy, wrapping this forward, it's important we have a position. to take this again past our adversaries. so on that note i want to talk about compliance that you mentioned. compliance is the basis line. the adversary know where's we have to be. they read the same a little bli.

they go above and beyond. how do you see what we need to do to get the investment or to use that risk ratio or in the strategy forward to get beyond compliance. compliance is a base leaine. it's never enough. >> i'll start and push to greg. compliance is certainly not enough. we're not there, though. the vast majority i've been associated with a lot of cyber incidents over the years. every single one of them was through a known vulnerable that had a known technical fix. every single one of them. if everyone had gotten to compliance, those at least the methodology that was used for the compromise, maybe our adversaries would have still got want go gotten in. doing it every single day over and over again is to drive up the costs for the adversaries. make them move further ahead.

make them be more creative and more expensive. and that will start to at least get us on a playing field where we can actually challenge their abilities as apposed to having them come into the doors we leave unlocked. >> i'll add on that. when i was the director of the national cybersecurity and km communication communicatio communications integration center, about 95% of the incidents they were dealing with, i characterized as the root cause was careless negligent or indifferent people. made a mistake. the technology was there, but it wasn't necessarily properly configured. it wasn't properly installed, et cetera. but upon reflection, i'm fiending that i was wrong by just saying careless negligence or indifferent. i would have penned overtasked

to that. when you drill down to it what a lot of the root causes are, is we go out there and we chase the latest fad. we put out the technology that we don't properly leverage to its full extent. we don't necessarily invest as much intellectual capital into the people and process aspect of properly came back and operating the technology that's out there. so i think as we take a look at the -- where we stand today, as well as into the future, making sure we have a good balance between the people process and technology. it's going to be the key as some of the new and innovative technologies roll out as well. leveraging well the technology we already have. >> and if i can add, when we talk about the people, it's not just those of you here who are cyber professionals or those of you who are in the basement of your organizations doing cyber

work. it's throughout the organization. we need to be able to have a collaboration about the technology and about the processes and about the people with the senior leadership of organizations. you know, that focus is somewhat about management attention. a senior leader who is asking really good questions is going to help to focus the team and they're the ones that can help with the overtasking. they can add resources or reduce tasking in some way, shape or form. >> if you're using a phone, if you're using a computer you're a cyber operator, period. >> and you're a target. >> you're a target. >> we see the same thing in the private sector. the attention has to come from the board. the board has to assess the risk appetite and that has to direct the entire strategy where the investment is made. it's not about how much you spend it's about how it's allocated and you accept a certain amount of risk as in any other technology and practice. so on that i would ask, grant, you've talked about technology and modernization in other

venues as well. one of the issues we've had is we have very large complex systems in the government. from our experience some years ago, you can't rip and replace just because it's old and it looks bad you have some product from 2002. however, it does take a process. at some opponent that product's not going to work anymore. we have to start working now. that's what you have to do. >> we definitely recognize, we can't continue to maintain the stuff we have forever. it's just -- we can't maintain it both from an operational and a customer service standpoint, but we also can't support it from a security standpoint. we have a really big focus, this administration has come in with a significant focus on i.t. modernization. how do we enhance and raise up and modernize the i.t. we have. how do we do it in a way we're not building the next decade's

legacy systems tomorrow. we've got to do it smartly. the good news i think is technology is there now. there are ways and as we move towards more shared services, as we moved towards cloud services. as we make smart decisions to how we don't have the government trying to update an infrastructure stack. i think we can get there at the same time and i talked about this earlier. you know, our ability to update policies to facilitate the agencies to leverage those technologies is absolutely critical. we have to get ahead of this curve and stay ahead of the curve. today we spend i think about $90 billion a year on information technology. somewhere north of 70% of it is on sustainment. a lot of it probably is sustainment of legacy items. we've got to be able to tap into those dollars to fund the modernization efforts going

forward. >> anything? >> well, i think the cheese has moved for everybody. you continue to use legacy models of dealing with i.t. and recapita recapitalization. frankly, having been in the private sector now for the last couple of years, you know, there's some really radical ideas in the private sector including recapitalization and depreciation on your balance sheet. i'd like to see the government leveraging those common business practices we see. making sure that you plan for the obsolescence of the people and processes that work in tandem with the technology. making sure that we have that as

part of our construct is going to really help as we move forward. >> i agree. i want to shift to the content that we talked about briefly about the binding operational directive. that was start adfed a few year ago. if you could comment on how important those are. i always tell people this is not an easy -- it's an authority that gives dhs a chance to say all the agencies are going to do this. the authorities from omb to -- yeah. working in partnership with omb. it's important to know that's not easily done and it's thoughtfully done. when these things come out and tell people to read the advice and think about the advice because it's what the government's doing and it came from a lot of thought. if you want to comment on that? >> i'll start out by saying thank you to the homeland security committee staffers who listen to me talking about in the military our a commander would issue a fragmentary order,

tasking order in the like. when an order was given it was expected to be done. they brought in the legislation, the creation of the binding operational directive where dhs would gather the information, do a quick interagency coordinat n coordination. when something positively had to be done across the federal government from a cyberperspective. it could be issued through dhs. it was a step in the right direction. i think that we need to be faster and agile on that. in the military, you can make a decision quick and it gets done. but with the current binding operational directive process we've seen a lot of maturation since the act was originally put out in december of '15. it's important to have unity of effort. and having been in dhs, i was well-trained to say if you see something you should say something. and that's really been one of

the successes of the binding operational directive. to assess the risk, decide a course of action and get it out across the entire u.s. government. >> yeah, i would add that i think the binding operational director has filled an important void we had before. we had laws and policies and guidance and then every agency was sort of told to figure out what all that means and what to do about it and how to do it and apply it to their infrastructure. and all those things have to be lowest common denominator. they have to be the same for everyone in every enterprise. they can be more tailored, more focused and more specific. also i think really the value that's come out of the binding operational directives is the management attention that they get. because they go to the senior leaders of agencies. the compliance, again, are you

done yet is checked and followed up on is recurring depending on the operational directive. a recurring conversation. i think -- as much as i would say some agencies go another oh, my god another bod i have to comply with. once they really start looking at it, they go wow that made my deputy secretary have conversations with me they probably never wiould have. it created that attention that they may have been screaming about from the basement for quite some time and really helps us push that forward. >> and we're seeing now the private sector is looking at them as well. that subsidiary benefit is really paying off to better protect critical infrastructure across the country. >> it's also a good example of leveraging authorities at omb to help the right skill for the right job. to help the agency that has the information to put it together. in this case, cyber too, ask the

other agencies, mandate the other agencies to do that and level the playing field. when some have come out, many in the private sector has said does this mean anything for us. and my answer has been, again, those are thoughtfully written and necessary. look at the words the government's saying they don't mandate anything for private sector but it's very good information as those come out. as you sort of the ghost of sisso past and present, what advice would you have for the private sector or those running programs in the military or government? what advice do you have from this chair on how to work with you, and, b, help to change that model to a risk driven model, if not already, to get the needed investment. >> i would say probably two things. first of all, is really a risk management approach. talk about risk. talk about risk with your senior leadership. you know, we want senior leaders that are asking questions about,

you know, how are you looking at the risk of your organization. where are you applying your mitigations, what are your mitigations, where are you accepting risk, which is an appropriate approach in some cases. but really take that risk management approach going forward. and then i would -- so for the second one, a focus on fundamentals. they're aren't -- and many of you have perhaps the secret sauce or secret product that's going to solve all the woes. i haven't found it yet. but i think there's a lot of just doing our due diligence, patching our systems, using strong oauthentication. all the things we can do to have as resilient of an enterprise as absolutely possible. i would say focus on those two things predominately. >> i'll throw in another, too. it goes back with some of the fundamentals.

first, as a war college graduate i'm require today quote a dead prussian in every public appearance. i'll remind everybody that frederick the great said he who defend everything defends nothing. we need to make sure that we're protecting the crown jewels. so i think it's critically important to understand the value of your information and don't necessarily spend a gazillion dollars protecting a piece of information that perhaps is not worth that squeeze. so making sure that you're implementing proportionate defense with a firm understanding as to the value of your information. both classification and sensitivity of the information is critically important. >> so what keeps you up at night? in the cyber perspective. >> are there other realms? i think the thing that botherers

me the most is still the risk exposure that we have with our critical systems that are out there. the advent of the internet of things continues to expand the risk exposure and the price of entry for somebody to engage in malicious mischief and criminal activity, the price for them is pretty low. i see the threat landscape continuing to expand and risk exposure continues to be high. >> i think i'd say china. you know, and i could say nation state actors, but as far as an adversary that has, you know, displayed their intent, has clear means to get into and to attack, our critical systems, our government systems, you name

it. both from an intellectual property theft point of view as well as an espionage point of view. to me. that is as a nation, this isn't a government problem. it's not a federal cybersecurity problem. it's how do we protect, we become so dependent on our i.t., many of you are very dependent on it right fnow as we're speaking. yet it's also has the potential for just catastrophic impacts when it's compromised. so our ability to protect against your rogue criminal or kid in the garage that used to be a threat probably isn't anymore. it's the nation state actor and the particular nation state with the capacity and capability and intent is the one that concerns me the most. >> double clicking on that. in the job of federal sisso, how are you helping all of us to fix

that? >> so i would say what we're trying to do is we want the federal government to be an example. we should be setting the example for how organizations should look at cybersecurity. so to your point, you know, private entities should look at the requirements that we put upon federal agencies. they're for a reason. they're all there for a reason. maybe too many of you to ever get to but the ability to understand the risk of your environment. so, you know, we're trying to put tools out there for the country to lencverage and then want to set an example and implement them with directives, policies, through special pubs, you know, thru all the mechanisms and levers we have to protect your information when we're holding it in the government. but also to serve as an example of how to best -- you can best

protect your information as a citizen or as a corporation. >> okay. >> yeah, i agree with everything that grant said. we're running out of time. so i won't beat that horse anymore. we're all in this together. and i think that it's -- the former federal sibso, we were trying to get things done right and set the good example for industry and academia and everything else, all citizens. we're all stakeholders in this process. i personally want to thank grant and the team that's still on the watch for trying to make things better for all of us. >> many thanks to greg and to grant for the work you've done. the work you do and many thanks to all. thank you. [ applause ] >> thank you.

>> thank you very much. our last panelists, thank you for that interesting conversation. this fireside chat coming up now is a great segue to the last panel. this one is about harnessing artificial intelligence and machine learning and cybersecurity. the moderator is brad metarie. thank you very much for moderating this panel. please allow me to briefly introduce your panelist. as a programming note, on all

the introductions i'll be giving, i'll keep them brief because you can see it up on the board and the full bioes are available in your program. so those are the panelists. general jack shanahan, united states air force. he's the director, joint artificial intelligence center, j.a.k.e. at the pentagon. the vice president of amazon web services. chief technology advisor to the principle deputy director of national intelligence. and lynn parker, assistant director of artificial intelligent from the white house. brad over to you. >> all right. thank you very much. and good afternoon. today we're going to be talking about harnessing artificial intelligence and machine learning and cybersecurity. today, there's probably no

bigger business word in the industry than artificial intelligence. we had the black cat cybersecurity conference a few weeks ago. everyone should be rest assured there's at least 3,000 a.i. cybersecurity companies as of last count. our objective for this panel is to talk about real world applications and really demystify a.i. so just kind of diving in, i wanted to talk about a.i. has gone from a very technical term over the last few years, into something that's prevalent now in our program. in our programs. and, you know, a.i. is more than just building an algorithm. what are elements to developing a successful a.i. program? dean, you want to get started? >> so building a successful a.i. program.

a.i. is technology, but it's technology informed by people and process. i guess number one is you have to have the people with the skills in order to do the job you're asking them to do. and that means that we need, you know, from where i sit in the intelligence community, we naee to invest in the workforce. one of the examples i use from time to time, if you ask an average imagery analyst what they need and their job is to look at images that are collected by satellites and clari classify them by what's in them, they want a bigger monitor or faster computer under their desk. what i generally mean, what is going to fundamentally change the way you're doing business tomorrow so you don't have to count airplanes on runways. the same issue is true with cyber at large. we need people who understand the promise of the technologies we're building.

that know how to apply them to our particular problem and know how to know whether they work or whether they don't work. one of the fundamental challenges we all have in a.i. and machine learning today is the idea of assurance. how do we know when it works and when it doesn't work? that knowledge is really, really important. the cloud computing technology has produced, but we also need access to the technologies of ai. so gpus are the most orve. but not only gpus, but gpus more for processors, data rays and whatever else the brilliant hardware engineers of the world are creating to accelerate these technologies. you need access to the digital foundation. third, you need data. you have to have data. it's the cure rated data that is tagged properly and formatted properly so it can feed machine learning. we need processes to create and

collect that da a ta and lastly you need mission. you need the consumers and the mission to be telling us what their problems are so we actually can go after. technologists can build solutions for anything. we need to know what the problems are. stated in a way we can apply the technologies. >> you're standing up the joint ai center. i know you're working a the lot of initiatives. as you're looking at stretching your programs, what are some key things you're considering? >> everything dean just said and then a lot more on top of that. if you were to break down in any ai program, machine learning, typically our focus area right now, the three common threads

whether it was an industry or in the defense department or the intelligence community would not surprise you. talent, culture and data. and i can reverse the three words in any order. those are what i dole with every day. and the data challenge is a particularly hard one for the cyber piece. >> let's pull the thread on that. i was at an event a few months ago talking about other nation states, and our adversary have the gift of data. i thus one of the things we have been struggling with is how do we bridge the gap between the government and the developer, silicon valley and the community to provide the data they need to build and tune algorithms. how are you seeing us start to bridge that gap? >> a couple different thoughts on that one. first of all, the conversation we're just having in the green room before coming in here is

the difference of an amazon or google or microsoft, the companies build their data in a certain way from the very beginning. the challenge is whether it's in the intel community or department of defense, we didn't build our data expecting a future of artificial intelligence. we have to look at what that world looks like to train against the data, integrate the models into the systems that were just never meant to have ai build them. so it's a a range of problems. to your other point. i was talking about this last week. the fact that a china has access to data, which is a very common talking point of china is leading the way in adoption of ai and also in just data. data for what? it goes back to what dean was saying earlier. data for what purposes. what do they intend to do? if i'm collecting social media data for the purposes of a social cred score, does that help me field a full motion video model for detecting, tracking, classifying objects on a battlefield in the pacific or

middle east. the answer to that is no. are they learning lessons, yes. but just data by itself is a starting point. and we can go into a lot more detail on the challenges we have of just getting to the data part of it before we bring in a model to try to assist the utility. >> so talking about good discussion around programs and what it means for success. when i think of amazon, i think you guys have a lot of data and you're working to optimize and lean out a lot of your om and other functions. i want to spend time talking about some successful use cases and applications of ai in the sign r security world. you want to get us started and share some of the initiatives and programs that you're working on? >> sure. so first, i want to echo what both of them said about what it takes to buld a successful

program and making ai adoption. if you look at actually what changed. deep learning is spurring the ai revoluti revolution. it was written more than two decades ago. it's basically -- always been hungry for specialized computers and it's a huge amount of data storage and access and actually making it easy for everyday developer to use it. this is where things like cloud has come in to change and that's why ai is experiencing a renaissance in the cloud where an everyday developer can have access to where they can get computers on a permanent basis and get a huge amount of storage on a monthly basis. now with this, we are seeing not just a.i. being adopted in high-tech industries all of the way from let's start like cybersecurity and the example of customers like new data with the

machine learning services and they're able to, not only have the machine learning deployment and development time with more than 60%, and they were nearly able to stop up to 100% of their credit card transactions with a bank. they were able to use computer techniques to actually address like fishing attacks and now not just in cybersecurity and now it is the pharma and also in financial industries like intuit and the transaction risk, but the common thread on what it takes and not just in amazon, but in other companies that are first. you need to buy in.

to a large extent, if you're a cio in a private sector they're a major stake holder in the public sector, there is an element that i tend to obstruct ai like a black box that you're not comfortable trusting, but tell the personal story of amazon, in more than five to ten years and amazon with the leadership team realized that the machine would transfer not just the tech part of the company, but every line of business and they're in sales or marketing or pricing. so they mandated something that every team has to answer and this was more than five years ago that they actually had in their annual planning session and what is your machine learning strategy? within parenthesis, they said no, that's not a good answer. triagain. so this forced every executor to think about what does machine learning do? what should be my machine

learning strategy, and what are they going to do? so that's when we created a machine learning, and so they'll get trained on various gardens and techniques and then finally we actually had a strategy for collaborating on data sets and held customers and ourselves with annotation and data cleanup because the dirty secret about ai and machine learning is while we hire the scientists to build machine learning algorithms. more than 50% of the time they do data wrangling. you'll probably agree which is kind of weird when you think about it because you expect them to work on the latest and greatest models and they spend so much time on data. this is why when i talk to cios and the stakeholders and public sector, they have the buy-in and get the strategy working well and then the third one is a talent and they're skilled in machine learning and that's why

woe have amazon, and now we make it available for free so that they can get trained. this is what we see across a wide variety of industries altogether. >> for other panelists, what are the other use cases that we're starting to embrace from the federal government? where are we seeing some success stories? [ inaudible question ] [ inaudible question ]

[ inaudible question ] >> i know there are a couple of cybersecurity use cases that you're starting to explore. can you talk about what you're seeing from a trend perspective there? >> it wouldn't surprise you to have the starting point of that be data. you could make some analogies to

project maven as a pathfinder project where we spend a lot of our time on the front end, object labelling and preparing the data. 80% which matches pretty much every project that i've seen is you spend 80% of your time working on the enablers and they do break down a little bit in cyber instead of going out in labeled objects for full-motion video and there are known objects on the ground and we have an ontology where people, buildings and vehicles and we work down from there and cyber is a little bit different problem to begin. what does normal look like? what is the baseline of normal? i have to know what baseline is and much more challenging on cyber than it is in a

full-motion video and our humanitarian assistance to relief case, so if i go back to starting with the data problem on cyber. it's the most basic problems that everybody begins with and data access and data quality, and data content and data classification and data format/standards and you can go in different directions on that. so what we had to do was reset a bit and our challenge is without getting the technical details of this and we have 24 cybersecurity providers and all of whom are collecting data in slightly different ways. so our starting point is coming up with the cyber data framework coming up with the cyber, and to come up with a starting point with data curation and content and sharing and storage. just on that agreement, i think we'll have much more success down the road as we bring in commercial vendors to bring product evaluation. they didn't quite know what data they were going to be seeing and there is not an image net equivalent for a number of different reasons and we'll talk about that separately and we'll have to come back to ground zero on this and our first of three lines of effort is what we're calling event detection and the third one is network mapping. all of those have the same basis of a data problem. so by going back to the beginning on a cyber data framework which is nothing more

than could we agree on a common set of procedures from now on on data coming in. if that's not the starting point we don't have the decades worth of really nice, clean, curated data which even swami was saying that's not entirely true of any of the companies and it is much more true than it is for the department of defense than i would say for the intelligence community. >> and he made the point earlier that every cybersecurity company is now a cyber a.i. company, and i would make the point that within the last decade many companies started branding themselves as cybersecurity companies and that gets into the definition of what problem are we trying to solve, right? >> a decade ago we talked about cybersecurity we were talking about antivirus definition, right? now we're talking about a living, breathing ecosystem of the world and as general

shanahan said define normal? how do i even know the difference between what's normal and what's abnormal so i can detect anomalies and we simply don't know. we actually don't know the answers to those questions right now and that makes it challenging to develop solutions. so this community here, this cybersecurity community needs to be thinking about how do we know what's normal? how do we detect a variance in the system? how do we make sure that our systems are appropriately secured against cyber attacks that we can't get defined, and that fundamentally is the challenge. ai can help with it, but ai is not a magic bullet. it's not jack's magic bean, right? we -- we it can solve some problems really, really well and other problems and particularly the kinds of ai that we're talking about now, the machine classifiers and so on. you can solve those problems and not every problem boils down to

that problem. >> one of the pitfalls i see many customers fall into the hype or the expectation trap. ai is not a silver bullet by any means and you set out, and the best way to go is you start small and actually you reiterate and check to see how well it's a problem and continue to trade. it's almost like a journey that you're going to be on and actually not just months and years to come and you're absolutely right and you're going to find a project and it's going to be big and it's going to be massive and how we're doing in six months to a year and if not it's by definition your chance of success will be low and you're absolutely spot on, and this is something, it's almost like a journey of discipline how you had to progress. >> if i can add this as well on the data piece. it's not just trying to wrangle it into a good form and it's

also determining whether or not you can trust it and that gets into some of the challenges with data poisoning attacks, for unstance where you may have perfectly good-looking data, but in fact t may have been tampered with in some way and so that's another challenge on top of just the quality of the data that we have from a formatting or curating perspective and has someone actually tampered with it and so that gets into rnd challenges on how to make sure that the data is pristine and the way you intended for it to be and it's not included within that, perhaps some examples of how you're learning unwillingly that a particular data set is not -- either is or is not indicative of some sort and that's an extra challenge of not having the data or not having good quality data. if you have that, can you trust

that you have good data. >> and this idea of a trustworthiness, the data is really critical and you can imagine in the business of intelligence, our job is to see over the horizon with enough time to impact the difference. well, in an era of adversarial networks producing deep, fake videos and fake text and fake audio and being able to substitute anybody's case on anybody's video, yeah. there are power tricks right now, but they have, you know, if you look down the road, it has the implication of it being very difficult for us to separate truth from fiction, and that makes the job of intelligence really, really hard, right? because if you don't know the difference between truth and fiction, you've got a big problem on your hands so the kinds of things you're focused on in the intelligence community what's real and not real, really, really huge. it's as applicable to the cyber

do main in which we look at these problems. >> so -- based upon the previous conversation, we're starting to address some fairly basic use cases and we're starting to move towards adoption. you have a captive audience here. in terms of research and development, i would like to hone in on new ideas and where this community should be investing for the future. dr. parker, do you want to start us off there? >> sure. when you think about ai and cybersecurity together, there's ai for cyber security and there's also the cyber security of ai and both have important challenges to them. you can imagine using ai for cyber security and doing things like being able to understand your adversary and trying to understand how they're attacking and have behavior and past history and use that to predict

what future attacks might look like, for instance and that's an interesting challenge for the ai and cybersecurity. the other direction for cybersecurity for ai looking at challenges like how do you make sure that a model that an ai system learns is not reverse engineered to somehow detect sensitive data or information that you don't want your adversary to learn about. i mentioned data poisoning attacks and there are a number of other of these kinds of challenges that you want to have your assistant to be trustworthy, so that you can ensure that when you use it it will do exactly what you planned for it to do, and that is in and of itself has a lot of rnd challenge e as well. the national science and technology council every three years put out a national or a federal cybersecurity rnd strategic plan.

so they're preparing that plan now to be coming out this year and it will outloon a number of the federal government will be investing in. >> so for the intelligence community, i encourage you to go to the website and download the strategy and augmenting intelligence machine, a.i.m., and it's not to augment the intelligence and it's to augment their activities. that strategy says we need to do four things. it says we need to invest in the digital foundation, the data and the compute. it says we need as government to be fast followers because we're in the interesting position as a federal government for probably the first time since the second world war, we are not the leading investor in the technology area.

in fact, we're not each the minority investor. the economy is the investor. in 2016 mackenzie estimated that there were $50 billion in global investment in a.i. and machine learning and they estimated that there was a billion dollars in u.s. investment at that time and 50 is in the billions and yes, we're spending more since 2016. the d.o.d. has announced their strategies and we don't publish our investment, but you can imagine that the private sector investment has accelerated it and it's far exceeded government expectations and we have to be fast followers and adopt the technology of the world. next, we have to invest in the gaps and we have to invest in the things that the private sector hasn't been invested in as we are. so think about a bell curve. where is most of the private

sector? the middle of the bell curve where your shoppers are, dollars, click, ads, eyeballs. what's the general's problem? what's my problem? low probability, high-cost things happen out there and that's not where the majority of the investment has been made and that's from i need to invest. >> our we need to be investing in long range and understanding and semantics and meaning and knowledge because ultimately counting air points faster is good, but it's not good enough i want to know why the planes ran yesterday and why not today? because ultimately the job of intelligence is to understand that. >> yeah. i'll quickly add a thing from the private sector respect. and we tend to use day one even though we're 20 years old and that shows how we tend to think. in the machine learning world it is so early and yes, it's day one and we've just woken up and we haven't had a cup of coffee yet and it's that early in terms

of how much early we are in this game and there's so much rnd that there's still more to be done and we have the internet and the early '90s and so forth. so in terms of what we need to see in rnd and it's not accessible and getting data done and there is in the machine learning models and so when it produces a result, what we see even with a health care customer is the consumers of these machine learning models, hey, you're scheduled for surgery and you may want to take and it is optimized and you may not trust the result and historically, if you had done this you will be 40% efficient and so forth. so there is even these elements of explaining these results so that people will trust it more and it's going to be a lot more important and these are some of these areas that are still in

research to me and we have to invest a lot more, not just in the private sector and also with academia and there are aspects, as well and be a partner on nsf on many of these topics, as well and fund specific programs. we'll continue to do more. >> just to the cut to the chase, it comes to this element of trust. if weigh look to a future of more fighting or defense of which where we're no longer measuring actions, counteractions or seconds, but milliseconds and microseconds and trust becomes the sine qua non, and it's a pristine lab environment and doesn't work in the cases that dean mentioned in a very dirty dod environment and the idea of proving that it can

work under those conditions and that's a partnership and give being able to perform in those instances and i would just go along with that and say we need to be thinking about ai and a red teaming approach and automating the teaming actions to think about the contextual things behind the scenes and counter a.i. is what we're dealing with and it's analogous and counteraction and that is something that is upon us now and we need more thought in the commercial enterprise. >> that's a very interesting observation and it identified two new and if you imagine the future of combat and the adversarial a.i. and how we'll adapt in the war fighting demand and certainly exciting times.

we have about a minute left and let's go around. each person has 20 seconds for any parting thoughts. dr. parker? >> certainly, if you look at the president's american ai initiative that was signed in the executive order that happened in february, there were a lot of these issues that are front and center and the rnd issues and trying to make sure that we have the people that we need in the ai space which includes the ai applied for cybersecurity space so that we can be the lead in these areas. you look at data about making data more available in rnd with cybersecurity and there are a number of these key areas that we touched on that the federal government is taking a number of actions to try to help the nation move forward to ensure and maintain american leadership in ai going forward. >> actually, we're just about at time, so to the panelists, thank you, and good discussion today and i appreciate everyone's time. thank you. [ applause ]

>> thank you very much, brad and the members of the panel for a great discussion. the next panel is preventing a cyber 9/11 and joining billion stage is jeff brown, chief information security officer for the intercontinental exchange in

the new york stock exchange and the honorable karen evans and assistant secretary for cybersecurity and emergency response at the department of energy. so bill, over to you. >> thanks, everybody, for joining us. to start off i would like to let each of our panelists and i know we got a brief introduction and talk about the current role and what they're doing in the area of critical infrastructure. so, jeff, if you want to start us off. >> thank you for having me. just a quick correction when it comes to intro, jeff brown head of something called cyber command and chief security officer for the city government of new york. we have the mission to defend all of those technologies that deliver via technology services to new yorkers each and every day and we also have a mission to bring cybersecurity to new yorkers and through solutions and awareness in ways that helps

them navigate away from the threats that they might encounter on the internet. to your question -- to your question about how we think about critical infrastructure and we as a large city government have parts of the portfolio agencies like the department of environmental protection that has ics, o.t., water services and new yorkers rely on and we also think about the criticality of things that are deemed critical services and new yorkers have to rely on with the 911 environment and that's how we think about it. >> hi. i'm karen evans, and i am the assistant secretary for cybersecurity energy security and emergency response, otherwise known as c.e.s.a.r. and it relates to all hazards both natural and man made.

so the emergency response function is really high right now on our efforts of our team due to the hurricane so i have hurricane responses. i have cyber responses i have the energy security piece. i have gmd, emd and we are responsible, if you're familiar with the national response framework, we are the esf12 coordinators under that with for our sector-specific roles and we also have specific authorities that are designated to the department of energy under the fast act of 2015. so i think i'll stop there, and take it from there. >> i'm carey rahm, vice president of product management. so i am fortunate enough to get

around the world and talk to a lot of different cybersecurity teams and help them with their incident response and the deployment of different analytics tools and we provide a platform that allows incident responders to investigate things differently and roll out different tools to defend the network and very interesting insights that i hope i can share in the panel as to what we're seeing and what we see some of the best practices in the cybersecurity teams as of today. >> i'm the real information security officer of the new york stock exchange. a little mixup early on. i work for intercontinental exchange and we're a global provider of financial market infrastructure and in five different cases over three different nations, we designate critical infrastructure and that

happens here vie at department of treasury in particular and i like to secure that side of the house. >> awesome. so let's start talking briefly about what the threat landscape looks like right now and what are you tracking in terms of threats for your infrastructure to your organization. karen, do you want to -- >> sure. mine's really easy. we can take a poll here of the audience, but anybody who has read the dni worldwide threat assessment, not that i have this memorized, but at the bottom of page 5 it talks about what is happening with china and how china is dealing and the capabilities that they have in the energy sector as it relates to oil and natural gas and at the top of page 6 it talks very specifically about russia's capabilities into our critical energy infrastruckur and what

they're capable of doing. so we're very focused on what the nation states could do. i don't own the infrastructure and it is all owned by private industry. so it would be good for us to talk about the trisector work that we're doing and how it relates to the national cyber strategy that was released by the administration. >> when you talk about a nation state attack, what does that look like? what's the nightmare scenario in your mind. what do you spend the most time thinking about in that landscape? >> i'm thinking about it right now. we have a natural disaster happening coming up the coast. we're worried about making sure that we can keep the power on and pre-positioning and working with our industry partners and it is all reliant on our industry partners and that's probably when we're the most vulnerable. >> interesting. same question to you. >> i can build on your answer. when it comes down to it, though as i noted before, there are things that fall into the traditional critical infrastructure category operated by the city government of new york. when it comes down to it, new yorkers rely on a whole ecosystem of providers. there are energy companies and

there are each and every piece of that critical infrastructure portfolio that makes the city run. i think when i think about the threat landscape what i'm looking at is greater connectivity and smart metering and smart services that a city needs to have guidance over, but perhaps not ownership over, and the way we have guidance over is we build better private-public partnerships and we get to be in conversations with providers because everyone has the best for new yorkers at heart and that's how we think about approaching the future. >> jerry? >> one thing that's really challenging in all of these roles that we have is defining the taxonomy. so when you ask about a threat it's kind of a double-edged sword. on one hand we can answer with almost anything, but on the other hand we don't get very specific. we mention threat actors and we

mention threat vectors in that, and when you think about insider threat versus a specific nation state and an objective, it's just a big soup. so what we've done is we create tax objectives which is what we found to have the unique buckets and what are try to do who they are and there are only three in there that have to do with data, and i think the most unique thing about the threat when it comes to critical infrastructure is that it's not all data like it is in the news and most of the consumer facing cyber threats and the ones that are data are intellectual property or pii or even non-public material information and the rest of them that are important to critical infrastructure, number one, sabotage and it's important to track it differently and not maybe because there are different threat actors and there are certainly different techniques that are effective asser have adversarially, and data know ma manipulation is the one we worried about. >> you were talking about tactic, tools and procedures of the adversary and you work

backwards from there essentially? is that how you approach that? >> that's right. the threat objective and -- and it's a good construct because it gives us a chance to talk at the board level about the whole ecosystem and it looks like you can take out saudi aramco and sony and very different companies and threat actors and everything else and we're having the same conversation about how it manifests and some of the ransom attacks were about destruction and not about extortion that would fit there as well and that's helpful at the board level and it's helpful to take the pii attacks and say y we know what that's about and we've discussed this before and where does that fall? so to set that priority at the board level is helpful and the stepping back, for us that means let's go straight to red teaming

and what did it look like when it happened elsewhere and that's where we gauge the residual risk of those. >> gotcha. >> you have a different perspective because you work with different security teams. are there any trends you observe across the customers you work with in terms of the threats they spend observing in the critical structure space? >> yeah, yeah. thank you. i would say the trends are more on how they're dealing with the threats and how the thought process is changing. so we're seeing some of the advanced teams that we work with going from truly defense top strategy to more of a okay, i know that there is a high risk of being breached. let me put the processes and procedures in place to make sure i can deal with that quickly and i can work with the downstream impacts before they can take effect and i can understand the full extent of what actually happened and i'm seeing them putting recording infrastructure to record everything about their environment and that's the first thing and being able to see what actually was impacted and what was touched and right down to the network data and being able

to respond quickly with different tools and techniques by being able to have an approach if there was an impending attack and they need some sort of new tool or a new innovation that they can apply and we're seeing that as a general trend, and seeing it as having a lot of good effect. >> gotcha. i want to drill into something, karen, that you worry about the threats to infrastructure that you don't technically own and that's interesting as a model, sort of it's not your fault, but it could be your problem kind of approach and what administrative constructs do you have to put in place to handle those things. what if x behavior or x set of infrastructure, there's going to be a problem and there will be

attack against that and how do you handle that organizationally? >> part of the -- i'm glad you asked that because you'll want to build off of this, as well. >> yeah. >> we talk often about public-private partnerships and i have a deeper appreciation, specially in the role of what private-public partnerships mean, because the only way i'm going to be successful to your point, is if the public-private partnership is there so i can convey from the approach that this is what is envisioned so that ssa does so they see value with what we're doing. the only way i can do that kind of analysis is they're contributing to the analysis capability so that we can say this is contributing to it, this is what's happening so that we can bring what you need to the government to bear, so we have a

whole government approach and we're only one critical infrastructure, right? under the dhs umbrella. so we have the whole of government, but i have to convince them that this is within the risk models that they have, the risk registries that they have, and the way they are doing things and our models are so different, but i would say that there is a huge trust model and a huge partnership between what is happening with the department of energy and the entire energy sector that if we were -- when we share that information they really listen and so it's incentivized that we need to do this to keep the lights on because we're such a critical need for the nation and the community all of the way down to the individual customer. >> gotcha. yeah, if you want to build on that. i know you were talking about similar themes? certainly. i think what it comes down to is addressing the domino impact that can happen based on the types of cyber attacks that

we've observed over the global landscape in recent years. when you think about 9 million people over five boroughs within the geographic confines in no, the reality is when you bring together public private partnerships and you have the right people sitting at the table with the right interest line, everyone recognizes that if one person in that diagram fails, the dominos start to fall and then from a business context, even though i represent a city government i think it does resonate with the private sector partners because you say unless we together pool resources, et cetera. when you are carrying my failure on your balance sheet as a risk because of that shared risk and you have cybersecurity and effort and you start to look at ways that we can address these problems and practice together and we've run a number of exercises and our hope is that it will help us to not only prevent, but then, of course, respond together.

>> and you generally find that they're receptive? >> awesome. and then, jeremy, do you have thoughts on that? >> i know yours is slightly different. >> well, you know, one of the things that would help for the sake of the audience is you start with the title about cyber 911 and when you are close to home and you can make the pivot over to things like power and transit and all of the implications it could have. on the economic side, i think it's worth just throwing in the scenarios that we're thinking of from a critical infrastructure standpoint and there are a lot to do with undermining confidence in the global markets and it's important to add that context and what does sabotage mean? i think it's important for private companies that are responsible for critical infrastructure to remember that that is not about the balance

sheet and it's not about the quarterly performance anymore. we have regulators that have different specific agenda that they're trying to protect, but when it comes to things like the department of treasury and the domino effect that that would have all of the way through every sector immediately. so it's not different in many ways, but in many ways it could be like splitting hairs. >> sure. so we're sort of talking here about the importance of developing close partnerships with people in related to the threat model and infrastructure and that goes to the broader theme of how are you gathering intelligence with these threats and who are you partnering with efficiently. is there anything that you guys do in your area and how are you getting that around the critical infrastructure? >> the threat intelligence is anyone who has lived through this saw it ten years ago, threat intelligence was so hot, so to speak and it was almost a buzz thing.

if you've been around for a while you might think let me wait a little bit and see if that ends up being a fad before we invest in it and so, you know, in our organization we consumed external sources early on including some commercial sources and later, we added the formal capacities that are handling going through that, but one thing that helped us get ahead of all of that is the isack and the information sharing analysis center and the fsi for financial services in particular, it really started and that is the embodiment of private/public partnership and it's a conduit between public intelligence and the private sector and more times than not it's actually peer to peer sharing among the members there that bears the most fruit and that did evolve into some automated and mechanical shares

so we have protocols for the sharing of threat intelligence and now we have systems that actually manifest some protections around that when they consume the intelligence and even what i call the narrative intelligence and the different banks and utilities and they reported a service attack and is anyone else seeing it and they're so helpful because intel feeds so many pieces of the life cycle and we think of the warning system and what's coming next and it arms our red team so the intel somewhere else is what we'll use to emulate the threat and the more detail we have the more

accurate it will be and it informs our controls and it informs our vulnerability assessment so we can prioritize if something is actually targeted. >> interesting. >> so sort of forward looking. so we have a good picture of the threats you're seeing, where are you spending most of your time? i know we talked about public/private partnerships are there others that you are trying to have for a defense apparatus for critical infrastructure? >> you have to think about the expanse of what a city government means with d.c. or whatever it may be. we have emergency management programs and so we're learning very much is the more connected we are into the whole apparatus of government capabilities and all of the teams that the emergency management can bring together to be proactive and exercise, there is a need to respond. a lot of times with the cybersecurity professionals and at times we may feel alone in

the fight and i think it's useful to bring back to organizations whether public or private the simple fact that if you talked to the people that are the enterprise risk managers, but have portfolios encompassing continuity of operations, et cetera. there's more capability to make sure that those services that the entity provide are resilient, reliable and can recover with peace. that's where we're seeing the trade craft now building from the state intelligence backgrounds and it's very heartening. >> gotcha. >> so i'm -- i'm going to -- you guys would be disappointed if i wasn't a little controversial here, and so we're looking at it a little bit differently, again, because i have a research and development piece associated with my office and of course, we have the national labs within the department of energy. so we're really looking to shift the paradigm, and really look at the framework, right? it has the circle and it talks about detect and protect and a lot of the stuff we're talking about today is in respond and the resiliency of how to recover. so i'm trying to change the paradigm and what the secretary has envisioned and what we believe will provide value out to the industry as a whole is we have efforts called the grid

modernization initiative which is modernizing the infrastructure to build the resiliency up front and to have self-healing capabilities to go forward and to change the dynamics instead of us spending research on response and we're spending a lot of research on how do you use smart technology and defined networks so you can then deploy these in a way that the system is detecting so that we can protect and then respond when we need to. so the other part is that we're not especially in our area as focused on information technology. so a lot of the stuff that you talk about today is very i.t. focused. we were focused on operational

technology and you mentioned industrial control systems and it's the nexus of where people are trying to gain efficiencies by using cloud to maximize that capability that comes from, okay, if we can gather this data and analyze it, like, that's what gets exploited. the more interconnections that happen, that's where we become vulnerable. so we're focused on that and then how do we secure and how do we have self-healing operational technology environments because that's, like, the i.t. world and you can look around this room and you guys are focused on operational technology. that it works and you can detect who is in there and is it running the way it's supposed to and is that supposed to be turn off and on and is that an adversarial testing and can we detect it? we're focused on changing the dynamic. >> sure. carey are there capabilities that you see that they're trying to build out in response to these kinds of threats or -- >> definitely. i see swings in both directions, you know, some organizations are heavily focused on the defense

side, trying to prevent and other organizations on the response side trying to scramble and respond to the incidents that occur and it's about getting that balance right and it's about being able to roll out new tools very quickly to defend the networks and it's also about having the historical data about what's been happening in your infrastructure so that when you do see something strange, you can go back and track what's actually occurred over time and having that balanced ride is important because it allows you to then say i'm going to defend the network as best i can, but i'm going to have the infrastructure in place for the stuff that i can't defend against and i think we all know the key issue is you cannot build a perfect infrastructure that is, you know, completely robust. at some point the state actor is going to have the resources and the know how and the time and the skill to get into your network.

you need to defend and keep those doors closed as tightly as you can and you need the information and the systems there for when someone does get in and starts to wreak havoc. as you saw with the ukraine attack, these threats they hang around for a long time before they actually do any damage and that's a period of time of which we've got to actually find this behavior and find these strange occurrences and neutralize them before they actually do any damage and getting that balance right i think really helps us achieve a much more robust infrastructure. >> gotcha. jerry, you had talked about the importance of red teaming and a proactive control where you can sort of test your infrastructure based on the terms of attacks. to what degree is the objective of the red team informed by other attacks you're seeing and what are things you might be seeing in the future.

can you tell us how you guide them? on it's directly informed by the intel that we receive about the type of threat objectives that we're concerned with. in that regard we're lacking, right? we have an attack or something like that and pull out the ttp as you mentioned earlier and begin there, but the whole point of that is that it's meant to be predictive and when we talk about -- so my organization is i like to define it within the first line of defense and the second and they're both on the reactive side and everything on the second line, i'd like to start thinking of as predictive and the threat modelling and scanning and all of that is really meant to predict, otherwise there is no point to doing it at all and we wouldn't bother. sorry, i'm combining two questions in one and what are we focused on and going back to that at the same time and it's equally both sides of the house taking that intelligence and flowing it to the second line and then from the results of that, going back to the first

line of controls that we need to put in place tomorrow without a doubt, but the one pervasive theme on both sides is automation, without a doubt, and i, i always say i want everyone that reports to me to take my job, right? i want to work myself out of a job because there will be new tasks that come out and that aren't on my plate and likewise, everyone in my group really needs to be working through automation and there are other things coming down the pipe and they can't do what they were doing yesterday and what they have to do tomorrow. so when we wrestle with technology on both sides because automation is about technology in many cases, it's build versus buy and like any company, we struggle with that and my approach to that to date is successful and it's called builders' buying and we do a lot of prototyping inhouse and then we go to the market once we figured out the challenges and can see through the oh, yeah, anyone can do that. >> how long is that build and buy cycliblely go for? >> i know it varies and depends on what it is. >> at some point in a project we either say this is a great and noble cause. we're not scaling and it doesn't have resiliency and let's go to the market and by then someone's created it, has done a better

job or is eager to do so, but there's a niche and a small gap of things where it's not very marketable. a product that would only be useful to us and those are actually the most valuable things that we have and they're based somewhat on the basis and on our culture and one of the things i talked to the board about before is the title was things the board has done for cybersecurity, but not on purpose. we don't unwind them by accident. that could be whether you're b to b versus b to c and it has to do with the head count and your employee turnover and all of these things have knock-on effects with cybersecurity and when it comes to something like that, and that's great, jerry and we're not going to make any money off of this then we can make it in house. >> running down to the last couple of minutes and looking forward, how do you see, of the threat landscape and is there anything that you don't see now that you anticipate to start seeing over the next one, two, five, ten years? >> to combat more connectivity that new yorkers rely on, we'll see more across municipalities across the notion of cybersecurity for the public,

perhaps. we launched nyc secure which is our commitment to new yorkers that we would bring cybersecurity to them of choosing and we released an app and all of the places that we provide free public wi-fi. >> and and you'll see municipalities go towards the people that walk their streets and say let's help you make better decisions as you navigate away from threats and let's respect your privacy at the same time. >> cool. >> so what we see across the

board is the mix of energy and we are energy independent as a nation and with that means there are other vulnerabilities that come into that. so the department has announced an advanced manufacturing initiative jointly with our office of efficiency and renewable energies and it's dealing with trying to manufacture and foresee, how do we continue to stimulate innovation so the wind turbines and the solar panels and the ev cars and the changing the battery because all of those devices connect into the grid, and so we are really looking to

see how can we engineer those so we have a mechanism in place dealing with private industry so we can continue to be energy independent and take advantage of industries, knowledge and then advance it through manufacturing. >> i'm not going to continue to talk about the threats because they'll continue to evolve. in five years' time, i think there will be a much more coordinated approach to defending the networks and more platform-centric approaches where it makes it a much easier tank for you to roll out new technologies. if you go to rsa or any of these big trade shows you will see thousands of innovations, but can you take advantage of those? probably not, very difficult so we'll see new ways to roll out technologies and roll out defenses rapidly in an agile fashion and trying to catch up to where the bad guys are at. >> i think we'll see advances in

the identification phase and that will be critical in not just authentication now and just about every packet on the internet. >> i think we're about out of time and thank you, everybody, for participating in the panel. yeah. >> okay. >> thank you. [ applause ] weeknights this week we're featuring american history tv programs as a preview of what's available every weekend opt c-span 3. tonight we'll show you a university of pennsylvania class on 18th century power struggles among native americans and

european empires. it's part of a seminar for teachers. that's tonight at 8:00 eastern here on c-span 3. this week we're also airing book tv programs in prime time to showcase what's available every weekend on c-span 2. tonight the theme is science and technology with authors gary marcus, thomas malone and keling harding. watch that at c-span 2. c-span's coverage continues as president trump hosts a keep america great rally in minneapolis, minnesota. live thursday at 8:00 p.m. eastern on c-span. watch any time on c-span.org and listen free wherever you are use ing the free c-span radio app. the house will be in order. >> for 40 years c-span has been providing america unfiltered coverage of congress, the white

house, the supreme court and public policy events from washington, d.c. and around the country. so you can make up your own m d mind. created by cable in 1979, c-span has brought to you by your local cable and satellite provider. c-span, your unfiltered view of government. more now from a recent cyber security conference. in this next part the discussion focuses on models for public and private collaboration as well as enhancing cloud security. participants include the director of the nsa's new cyber security branch who gave an overview of the new department. this is just over an hour and 45 minutes. okay, good afternoon, everybody, again. welcome to the second part of our program. i'd like to invite you back and